A Universally Consistent Modification of Maximum Likelihood

In some models, both parametric and not, maximum likelihood estimation fails to be consistent. We investigate why the maximum likelihood method breaks down with some examples and notice the paradox that, in those same models, maximum likelihood estimation would have been consistent if the data had been measured with error. With this motivation we define doubly-smoothed maximum likelihood as a natural mechanism for adding measurement error without bias. We show the proposed estimation procedure gives universal consistency in independent and identically distributed data. Our method of proof is new. The same arguments can show maximum likelihood itself is universally consistent in a discrete sample space. It is shown that the asymptotic efficiency can be quite high even when the bandwidth parameters are held fixed. Practical guidelines for the choice of kernel and tuning parameter are also given.